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ABSTRACT 



This paper presents an alternative means of assessing 



faculty teaching that can be used by administrators. This method has been 
derived from in-depth faculty and student interviews, and its criterion of 
quality has been validated with empirical data through computer sensitivity 
analysis. The assessment process has been successfully tested in clinical 
teaching trials. Interviews with faculty have identified that their implicit 
expectations for professional development can be described in terms of 
technical skills, professional competence, and professional attitudes. In 
practice, lecturers explain these three skill areas to their students and 
describe how students will be taught and assessed. Students are asked to rate 
the lecturer on these three areas, and then the lecturer rates himself or 
herself. The indicator of effective teaching, that the students and the 
lecturer are working toward the same changes, is measured by the change 
expected from the students subtracted from the change expected by the 
lecturer. A "zero" is the score that indicates perfect alignment. This method 
promotes a positive teaching and learning culture by encouraging teaching and 
learning that promotes students' critical and evaluative thinking. (Contains 
31 references.) (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM 032475 ed 45 i 247 



A Method of Assessing Quality Teaching to 
Create a Positive Institutional 
Teaching and Learning Culture 



Author: 

Tony Bastick 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



Tony Bastick 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

. EDUCATIONAL RESOURCES INFORMATION 
y CENTER (ERIC) 

On his document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




2 

BEST COPY AVAILABLE 



V 



SCOPE OF INTEREST NOTICE 



A Method of Assessing Quality Teaching 
to Create a Positive Institutional Teaching and Learning Culture 

Tony Bastick 

University of the West Indies 



The ERIC Facility has assigned 
this document for processing 
to: 



1 % 



In our judgment, this document 
is also of interest to the Clear- 
inghouses noted to the right. 
Indexing should reflect their 
special points of view. 



SI 



Introduction 

Postsecondary institutions assess quality teaching by (i) student feedback ‘opinionnaires’ 
and (ii) students’ examination results. Ironically, these forms of assessment can reduce the quality 
of teaching to an inequitable popularity contest of ‘easy courses’ leading to ‘grade inflation’ and 
‘teaching-to-the-test’. The high stakes attached to assessing quality teaching, for institutional 
league tables and for professional promotion, makes these chosen forms of assessment highly 
influential in adversely shaping an institution’s culture of teaching and learning. 



This paper presents an alternative means of assessing Faculty teaching that can be used by 
administrators. It is designed to promote an alternative positive teaching and learning culture 
within the institution. This alternative method of assessment was derived from in-depth faculty 
and student interviews. Its criterion of quality has been validated on empirical data by computer 
sensitivity analysis and the assessment process successfully tested in clinical teaching trials. 



This assessment method explicitly promotes students’ understanding and professional 
attitudes, as well as their traditional skills, by operationally defining the assessment of skills, 
understanding and attitudes. It respects professional freedom and the inherent culture of each 
subject area by giving each lecturer the responsibility of promoting the culture of their subject 
through their teaching and assessment of its skills, understandings and attitudes. It allows faculty 
to adjust their in-course teaching and assessment schemes to the changing needs of the student 
body and minority students in their classes. The method results in a single administrative 
decision-point number that is a measure of quality teaching as it applies to (i) a particular 
individual, or (ii) a minority group or (iii) the whole class. 



Many organisational factors and resources affect the teaching and learning culture of an educational 
institution. This paper focuses on the use of assessing quality teaching as a means of developing a positive 
teaching and learning institutional culture. There is a business maxim that states “what gets measured gets 
done and what gets rewarded gets repeated” (Friend, 1972). In educational institutions what gets rewarded 
is (i) the assessment of faculty teaching and (ii) the assessment of student learning. It is how the assessment 
of teaching is done and how the assessment of learning is done that has a great influence on the learning 
and teaching culture of the institution (Beichner, 1994; Hake, 1998; Halloun & Hestenes, 1985; Halloun, 
Hake, Mosca, & Hestenes, 1997; Hestenes, Wells, & Swackhamer 1992; Hestenes & Wells, 1992; Sokoloff 
& Thornton, 1997). These three aspects (i) teaching (ii) learning and (iii) the assessment of both, are so 
inter-related that they should be considered together. 



Bastick, T. (1998, November). A method of assessing quality teaching to create a positive institutional 
teaching and learning culture. Paper presented at the 8 th Annual Conference of the Institute for the Study 
of Postsecondary Pedagogy - Creating alternative learning cultures: culture, cognition and learning, 
Fllenville, NY. 
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Unfortunately traditional forms of assessment lack integration and problems caused by these traditional 
forms of assessment negatively influence the learning and teaching culture of institutions. The common 
traditional assessment of faculty teaching is via the use of student opinionnaires. For example, Seldin 
(1984) found that administrators utilized student opinionnaires in two-thirds of 616 institutions surveyed. 
Student opinionnaires have been criticised as popularity contests, where staff reduce the standards of their 
courses and lead students to expect high grades in order to ‘win their vote’ (Greenwald, 1997; Greenwald 
& Gillmore, 1997; Howard & Maxwell, 1982; Marsh & Dunkin, 1997; Marsh & Roche, 1997, 1998). 
Faculty are expected to change their teaching in view of this anonymous feedback (Moses, 1996). Often 
these opinionnaires are anonymous because students fear retribution for ‘failing’ their teachers or might be 
thought to be soliciting favourable grades for favourable assessments of their teachers. Faculty feel 
uncomfortable in this one-sided contest where they can be anonymously failed yet have little influence 
over the selection of students they must teach. Some faculty feel that having to change their course emphasis 
in order to please naive students is an infringement of their professional freedom to teach how they believe, 
in their best professional judgement, their subject should be taught. This has been contributing to a negative 
culture of low standards, demotivated professionalism and mutual distrust for some years (Arreola, 1983; 
Cashin, 1983; Cherry, Grant, & Kalinos, 1988). For example, “What is called development, growth, and 
self-improvement today becomes the means by which decisions for institutional personnel management 
purposes are made tomorrow. Faculties become wary and suspicious of this double message involved in 
the evaluation system” (Mark, 1982, p. 168). 

The traditional assessment of student learning is via examinations and coursework assignments of 
various kinds. Here the one-sided game is against the student who has little educational recourse and so 
can only resort to complaint about the course and the faculty - even to the extent of litigation. To protect 
themselves from the ‘court case student’ faculty favour objective assessment that does not expose their 
professional judgements about the students’ work. Such objective assessments tend to emphasise Bloom’s 
lowest cognitive level of rote learning. For example, “McKeachie (1987) has recently reminded educational 
researchers and practitioners that the achievement tests assessing student learning in the sorts of studies 
reviewed here typically measure lower-level educational objectives such as memory of facts and definitions 
rather than higher-level outcomes such as critical thinking and problem-solving that are usually taken as 
important in higher education.” Feldman (1989). Adult students in professional subjects rightly devalue 
courses that emphasise only rote learning, to the extent that they are prepared to cheat in order to maximise 
their meaningless marks. Problem attendance is a feature of such courses - students get the lecture notes 
from those who do attend. Students learn to unquestionable do as they are told, not to criticise the views of 
faculty and to parrot what they believe Faculty expects them to regurgitate in examinations. Such is the 
negative effect traditional assessments of teaching and learning have on the teaching and learning culture 
of the institution. Research has indicated that these problems are in part due to misunderstanding of mutual 
expectations (Bastick, 1995). Faculty lack clarity in explicitly stating their expectations and relating these 
to their teaching and assessment of the students. The students misunderstand what is expected of them and 
are confused. 

What is a positive teaching and learning institutional culture? The literature indicates that is one that 
encourages staff and students to be independent critical thinkers developing the attitudes and values of 
their profession. One where students and faculty value and enjoy the work they are doing. One where 
faculty and students respectfully work together based on a foundation of mutual trust. If an alternative 
method of assessment is to promote these changes, then first, that method must accommodate the wide 
institutional variations in assessment preferences and aim to improve teaching and learning by allowing 
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those lecturers who use it to appreciate more fitting teaching styles and enable them to allow their students 
to adopt more fitting styles of learning. Secondly, an alternative method of assessing teaching and learning 
must resolve the misunderstandings and confusions about mutual expectations in order to avoid the problems 
that lead to a negative teaching and learning culture. Thirdly, an alternative method must promote a positive 
teaching and learning culture by (i) ensuring students and faculty understand each other’s expectations and 
(ii) by ensuring that students and faculty are both working towards the same expectations (Abrami, 
d’Apollonia, & Cohen, 1990; Abrami, 1989;Bastick, 1995;Bortz, 1984; L’Hommedieu, Menges, & Brinko, 
1990; Miller, 1986; Scriven, 1994, 1995). 

It is important to separate attainment and enjoyment so that student evaluations of course enjoyment 
are not misused as assessments of academic attainment (Hake, in press). Hence, the separate criteria of 
effective teaching used by this alternate method of assessment are to maximise (i) the academic attainment 
of the students and (ii) the students’ and the lecturer’s enjoyment of the course. The measurable indicator 
of effective teaching used is that the students and the lecturer are working towards the same expectations. 
The construct validity that the measurable indicator assesses the criteria is p<0.01 for both (i) and (ii) 
(Bastick, 1995) 

Interviews with faculty on professional courses have indicated that their implicit expectations can be 
described in terms of three abilities (i) technical skills - rote learning, assessed by the accuracy of reproduction 
(ii) professional competence - appropriate transfer of skills to a novel situation, assessed by the justification 
of appropriateness and (iii) professional attitudes - the integration of one’s life and work by one’s values 
and beliefs, assessed by demonstration (Bastick, 1995). Faculty can be assisted in making these expectations 
explicit and in designing coursework and examinations that offer opportunities for assessing these three 
abilities. This professional development can be expected to improve the quality of their teaching (Askew, 
Brown, Rhodes, Wiliam, & Johnson, 1997). It is their professional prerogative to decide, and justify to 
their peers and their students, the emphasis theyjudge should be given to each of the three abilities on their 
courses. These judgements will depend on the subject, its level and the professional inclination of the 
lecturer. For example, lecturers on B.Ed courses expect an emphasis on technical skills in the first year, 
moving to an emphasis on professional competence in the second year and a greater emphasis on professional 
attitudes in the third year. 

In practice lecturers explain to their students the three abilities and how they will be taught and 
assessed. When they wish to monitor the effectiveness of their teaching they ask the students to rate how 
they see the current emphasis of these three abilities and to rate how they would prefer the emphasis to be. 
The lecturer makes the same rating of the course. The indicator of effective teaching, that the students and 
the lecturer are working towards the same changes, is measured by the change expected by the students 
subtracted from the change expected by the lecturer. Zero is the perfect score on the total of the three 
abilities, and indicates perfect alignment. 

The alignment score can be calculated for individual students, minority groups or for special 
comparisons e.g. is the teaching more effective for males than for females. 

The method promotes a positive teaching and learning culture indirectly by encouraging forms of 
teaching and learning that faculty and students use to increase their valued assessment results. Namely, 
this method encourages teaching that promotes students’ critical and evaluative thinking, high standards in 
technical skills and professional values. The assessment method promotes a positive teaching and learning 
culture directly through student and faculty assessment processes, as follows: 
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1. The institution promotes academic freedom and professional responsibility by confirming the 
lecturers’ professional prerogative to decide, and justify to their peers and their students, the emphasis 
they judge should be given to each of the three abilities on their courses. This is reinforced by 
recognising an assessment process that lecturers control. 

2. The institution promotes professional development by assisting faculty in making their professional 
expectations explicit in terms of the three abilities in their subject area and in assisting them to design 
coursework and examinations that offer opportunities for assessing these three abilities in their subjects. 

3. Faculty encourage students’ critical and evaluative thinking, to the extent they can justify as desirable, 
by not assessing the correctness of professional competence, but by assessing the students’ justifications 
of why their applications are appropriate. 

4. Faculty explicitly encourage professional attitudes, to the extent they can justify as desirable, by 
assessing demonstrations of professional attitudes on course assignments. 



Generally, the development of technical skills is already well served by traditional methods of 

assessment. However, an interesting staff development programme would be to share methods of teaching 

professional competence and professional values. 
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